Resource configuration in multi-modal distributed computing systems

ABSTRACT

A method and system for configuring available resources in real-time to automatically accommodate the needs of the system user in multi-modal distributed computing system is disclosed. Information about the location or environment of a wireless device is used, preferably in combination with user personal preferences and past history to modify the behavior of the wireless device, including the selection of the most appropriate mode of interaction with the device and the activation of applications thereon as appropriate.

FIELD OF THE INVENTION

The present invention relates to multi-modal distributed computingsystems and, more specifically, to configuring available resources inreal-time to automatically accommodate the needs of the system user.

BACKGROUND OF THE INVENTION

There are many types of wireless mobile devices being used in the worldtoday including mobile phones, personal digital assistants (“PDAs”),hand-held devices, and combinations of these devices. Wireless transportnetworks and wireless local area networks allow electronic content toflow to and from these mobile devices. With the growing popularity ofmobile devices, mobile information access and remote transactions arefast becoming commonplace. However, mobile devices impose theirlimitations on the end user experience. For example, mobile phones haverelatively small visual displays and a cumbersome keypad input. PDAshave better visual displays, but have the same input limitations. Asdevices become smaller, modes of interaction other than keyboard andstylus are a necessity. One such alternative is the use of multimodalaccess methods.

Multichannel access is the ability to access enterprise data andapplications from multiple methods or channels such as a phone, laptopor PDA. The term “channel” refers to the different browsing platforms oruser agents that access, browse, and interact with online applications.Multichannel applications are designed for universal access acrossdifferent channels, one channel at a time, with no particular attentionpaid to synchronization or co-ordination among the different channels. Auser has an array of channels with which to access content, whichappears separate but functional and consistent. For example, a user mayaccess his or her bank account balances on the Web using Microsoft®Internet Explorer when in the office or at home and may access the sameinformation over a dumb phone using voice recognition and text-to-speechwhen on the road.

By contrast, multimodal access is the ability to combine multiple modesor channels in the same interaction or session. The methods of inputinclude speech recognition, keyboard, touch screen, and stylus.Depending on the situation and the device, a combination of input modeswill make using a small device easier. For example, in a Web browser ona PDA, a user can select items by tapping or by providing spoken input.Similarly, a user can use voice or stylus to enter information into afield. With multimodal technology, information on the device can be bothdisplayed and spoken. This can be especially important in automobiles orother situations where hands and eyes free operation is essential.

Multimodal applications represent the convergence of content—video,audio, text and images—with various modes of user interface interaction.This enables a user to interact with an application in a variety ofways, for example: input with speech, a keyboard, keypad, mouse and/orstylus, and output such as synthesized speech, audio, plain text, motionvideo and/or graphics.

The term “mode” denotes a mechanism for input and output to a userinterface. A user can employ each of these modes independently orconcurrently. Multimodal applications incorporate any number of modessimultaneously so a user can vocalize his/her name, type in an address,send a phone number from a wireless handset—all within the same session,form, and application context. The browser will typically let a userselect the most appropriate mode of interaction based on the user'ssituation, activity, or environment.

The different modes may be supported on a single device or on separatedevices working in tandem. When separate devices work in tandem, this istypically referred to as distributed multi-modal computing. An exampleof distributed multi-modal computing is a user is talking into a cellphone and seeing the results on a PDA. Voice may also be offered as anadjunct to browsers with high resolution graphical displays, providingan accessible alternative to using the keyboard or screen.

Multimodal applications are an improvement over multichannelapplications. Advantages of multimodal applications include: multimodalinterfaces improve the usability of data services such as weather,driving directions, stock quotes, personal information management, andunified messaging; Application Service Providers can offer users a widerrange of personalized and differentiated offerings using multimodalinterfaces; many call center applications and enterprise data servicessuch as account management, brokerage accounts, customer service, andsales force automation offer voice-only interfaces and multimodalinterfaces added to these applications enhance a users experience; withmultimodal interfaces, a user can easily access and enter information,especially when using small devices by combining multiple input andoutput devices; multimodal applications improve a users experience withmobile devices and encourage the growth and acceptance of m-Commerce; auser need not be constrained by the limitations of a particularinteraction mode at any given moment, for example, while listening toinstructions on a Voice browser, a user is constrained by the ephemeralnature of the interface; a user may wish to listen to the instructionsagain; multimodal interfaces give a user the flexibility to choose themost convenient interaction mode that suits the task and purpose; theycan also exploit the resources of multiple interfaces in order for auser to have an enhanced computing experience.

Users of multimodal interfaces, however, do face certain issues. Theseissues include ergonomic issues and appropriateness. Ergonomic issuesmay arise as a user switches from one mode to another, such asalternating between listening and watching. Appropriateness issues arein the nature of a user disabling speech input and output when thiswould be distracting to nearby people. Considering all of the variousissues, however, a user must still select the most appropriate mode ofinteraction based on the user's situation, activity, or environment.

SUMMARY OF THE INVENTION

In accordance with at least one presently preferred embodiment of thepresent invention, there is broadly contemplated a system and method inwhich information about the location or environment of a wireless deviceis used, preferably in combination with user personal preferences andpast history, to modify the behavior of the wireless device.

In summary, one aspect of the invention provides a system for system forresource configuration in a multi-modal distributed computing systemhaving at least one resource capable of being configured, the systemcomprising: an arrangement for obtaining information associated with amobile device within the system; an arrangement for obtaininginformation associated with the system; an arrangement for configuringsaid at least one resource based upon the information associated withthe mobile device and the system.

Another aspect of the present invention provides a method for resourceconfiguration in a multi-modal distributed computing system having atleast one resource capable of being configured, the method comprisingthe steps of: obtaining information associated with a mobile devicewithin the system; obtaining information associated with the system;configuring said at least one resource based upon the informationassociated with the mobile device and the system.

Furthermore, an additional aspect of the invention provides a programstorage device readable by machine, tangibly embodying a program ofinstructions executable by the machine to perform a method for resourceconfiguration in a multi-modal distributed computing system having atleast one resource capable of being configured, the method comprisingthe steps of: obtaining information associated with a mobile devicewithin the system; obtaining information associated with the system;configuring said at least one resource based upon the informationassociated with the mobile device and the system.

For a better understanding of the present invention, together with otherand further features and advantages thereof, reference is made to thefollowing description, taken in conjunction with the accompanyingdrawings, and the scope of the invention will be pointed out in theappended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart of a method in accordance with the presentinvention.

FIG. 2 is a block diagram of the implementation of a system inaccordance with the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

In accordance with the present invention, information about the locationor environment of the wireless device is used, preferably in combinationwith user personal preferences and past history to modify the behaviorof the wireless device, including the selection of the most appropriatemode of interaction with the device and the activation of applicationsthereon as appropriate.

The environment the wireless device is located in may be established ina number of ways, and may include the actual physical location of thedevice or the location of the device relative to other devices,resources, etc. For example, a number of wireless devices now includethe ability to determine its position using the Global PositioningSystem (GPS). In fact, the use of GPS is mandated for certain wirelessdevices by the United States Government, e.g., cell phones. Theenvironment of the device may also be established based upon theproximity to or use by the device of a wireless network. For example, bythe wireless device entering a local network (e.g., a Bluetoothpiconet). The location of the wireless device may also be based upon ascheduled future location. It should be understood the present inventionincludes the use of establishing the location of the wireless device inany manner, and not just in accordance with the examples set forthherein.

Once the environment the wireless device is located in has beenestablished, the behavior of the wireless device is appropriatelymodified. Behavior may be modified in any number of ways. It ispresently preferred, however, that modification occur based on acombination of two items. The first is whether or not a user hasindicated that a certain modification, whether it be the launch of anapplication or otherwise, is to occur at a particular moment. The secondis the preference of the user, as determined through past instructionsas well as the history of the modifications, including applications andqueries, that were launched by the user in the same or similar location.

Given the number of environments a wireless device may encounter overtime, it is presently preferred that any past instructions andhistorical modifications be associated with the following specificsituations: Office, Local on the road, Remote business site, Localbusiness office, Teleconference, Conference room, Commute, On the roadbetween business office and remote business site, Business trip, andAbroad. It should be understood, however, that past instructions andhistorical modifications may be associated with any number ofsituations, including particular situations favored by a user.Association with specific situations may be accomplished without userintervention, either by the wireless device itself or otherwise. In anyevent, it is presently preferred that a user's ability to associate anenvironment with a specific situation not be precluded.

The preference of the user may also be determined exclusively throughthe use of historical data without reference to an association to anyspecific situation. Doing so, however, increases the computationalresources needed at the triggering event to determine the appropriatemodifications to the behavior of the wireless device. Given the natureof mobile wireless devices, the availability of computational resourcesmay very well be at a premium.

As discussed herein, the information interaction needs of a user in amulti-modal distributed computing environment are often influenced bythe location, device, available resources and other environmentalfactors. The following are non-limiting examples of how such informationinteraction needs may be so influenced in accordance with the presentinvention. A user with an audio-only device such as a mobile phone maywish to temporarily use a display resource (such as a monitor in alounge or a conference room, or an in-seat display on an aircraft) tolook at video information. A user driving a car may be interested ingetting some traffic or weather information, and may be interested ingetting the information downloaded to an accessible device or may wishto make reservations for network resources to access the information. Auser entering a new building may wish to have information such asrestroom locations downloaded to a personal device immediately uponentrance. A traveler entering the airport check-in may wish to haveinformation about departure gate changes downloaded to an accessibledevice. A user may wish to have only urgent messages made available (byemail or otherwise) when the user is travelling, but want to have allmessages made available when not travelling. Similarly, a user may wantto skip over any messages containing material if the user is in a publicplace. A user may desire a device with audio to be verbose outside of avehicle, but not as verbose when the user is in a vehicle. There arealso numerous other ways in which information interaction needs may beso influenced.

In another embodiment of the present invention, the behavior of thewireless device is appropriately modified based upon actions initiatedby or preferences of others, such modification not having been selectedby the user of the wireless device, e.g., the modification having beeninitiated externally of the wireless device being modified. Others mayinclude other users of wireless devices within the environment or theperson or persons exercising control over the environment in which thewireless device is located.

Once such example is where the location of a meeting has been changed.In this circumstance, the requestor or initiator of the meeting mayinitiate a behavior modification of wireless devices to notify peoplethe meeting location has been changed. Such notification may include thenew meeting location, and the notification may be provided when reachingthe building, floor or room, depending on network configuration andwhether the identity of the attendees is known. If the identity of theattendees is known and their wireless devices can be identified, theattendees may be notified at the earliest possible opportunity, e.g,upon arrival at the building or earlier. Otherwise, all wireless devicesentering a defined environment, e.g., the conference room, may beinformed of the change in location.

Such a forced behavior modification may also be utilized in otherappropriate circumstances. Examples of such situations include emergencysituations, and may also include situations previously discussed withrespect to user preferences. Should an emergency situation occur andimpact an environment, either directly or indirectly, the person orpersons exercising control over the environment in which the wirelessdevice is located may provide notification of the existence of theemergency situation and optionally, instructions on dealing with theemergency situation. Another example of a circumstance in which a forcedbehavior modification may be utilized is when the person or personsexercising control over the environment in which the wireless device islocated determine a particular behavior is appropriate for all wirelessdevices with in the environment. An example of such behavior may be novoice interaction—which would be appropriate for a church, auditorium,or similar environment.

Referring now to FIG. 1, a flow chart of the method in accordance withone embodiment of the present invention is shown. At step 210, thelocation of the user is established. Once the user's location isobtained, at step 220, it is appropriately classified into any number ofpre-selected classifications or locations, which may include office,home, or meeting. Next, at step 230, the location classification is usedto look up the user's preferences for input, output, and associatedresources. Once the preferences are obtained, the appropriate input,output, and resources are activated. At step 240, the appropriateapplications are determined and launched according to user's preferencesfor the location classification. It will be appreciated that theinvention contemplates that steps and 240 may be interchanged orperformed simultaneously. Preferably, steps and 240 are performedsimultaneously. Additionally, the preferences at steps 230 and 240 maybe obtained from a database of historical keystrokes and queries or maybe obtained from a database of preferences for a given locationclassification. In at least one embodiment, the user may choose to havecertain input, output, resources, and applications launch based upon theday and time. Thus, it will be appreciated that the user's context maybe in relation to geographic location, day, time, available resources orany other relevant context for selecting appropriate input, output,resources, and applications.

Referring now to FIG. 2, a block diagram of the system according to thepresent invention is depicted. Upon first users' entry into the localnetwork, the information of the first user 100 is transmitted to thenetwork controller 120; alternatively, network controller 120 may obtainthe location of user 100 through GPS 110. Network controller 120accesses database 140 to determine the preferences of user 100 forapplications and functions to be launched based upon location. Once thepreferences are obtained, network controller 120 launches theappropriate applications or functions at 150. The network controller maythen store the history of this activity in database 140 to be consultedat a later time.

In another embodiment, network controller 120 accesses database 140 todetermine the preferences of user 100 for applications and functionsgiven the date and time. Once the preferences are obtained, networkcontroller 120 launches the appropriate applications and functions at150. The network controller may then store the history of this activityin database 140 to be consulted at a later time.

It will be appreciated that this same system and process may be madeavailable to user 200. Further, the geographic and calendar informationof multiple users may be coordinated by the network controller 120 so asto automatically reschedule or relocate a meeting based upon geographicinformation and resource availability of the multiple users. Theinformation about the new date, time, and location and the meeting maythen be transmitted to multiple users through the network controller120.

It is to be understood that the present invention, in accordance with atleast one presently preferred embodiment, includes an arrangement forobtaining information associated with a mobile device within the system;an arrangement for obtaining information associated with the system; andan arrangement for configuring at least one resource based upon theinformation associated with the mobile device and the system, which maybe implemented on at least one general-purpose computer running suitablesoftware programs. It may also be implemented on at least one IntegratedCircuit or part of at least one Integrated Circuit. Thus, it is to beunderstood that the invention may be implemented in hardware, software,or a combination of both.

If not otherwise stated herein, it is to be assumed that all patents,patent applications, patent publications and other publications(including web-based publications) mentioned and cited herein are herebyfully incorporated by reference herein as if set forth in their entiretyherein.

Although illustrative embodiments of the present invention have beendescribed herein with reference to the accompanying drawings, it is tobe understood that the invention is not limited to those preciseembodiments, and that various other changes and modifications may beaffected therein by one skilled in the art without departing from thescope or spirit of the invention.

1. A system for resource configuration in a multi-modal distributedcomputing system having at least one resource capable of beingconfigured, the system comprising: an arrangement for obtaininginformation associated with a mobile device within the system; anarrangement for obtaining information associated with the system; anarrangement for configuring said at least one resource based upon theinformation associated with the mobile device and the system.
 2. Thesystem according to claim 1, wherein said information associated withthe mobile device includes contextual information associated with theneeds of the user of the mobile device, the location of the mobiledevice, and the environment in which the mobile device is located. 3.The system according to claim 1, wherein said information associatedwith the system includes information associated with the capabilities ofdevices within the system.
 4. The system according to claim 1, whereinsaid information associated with the system includes informationassociated with characteristics of the environment in which the systemis located.
 5. The system according to claim 1, wherein saidcharacteristics of the environment in which the system is located arecurrent characteristics.
 6. The system according to claim 1, wherein theat least one resource to be configured is an interface resource.
 7. Thesystem according to claim 1, wherein the at least one resource to beconfigured is a computing resource.
 8. The system according to claim 7,wherein the computing resource is an application.
 9. The systemaccording to claim 8, wherein the application has multipleconfigurations and the application is configured to be most appropriatefor the environment in which the mobile device is located.
 10. Thesystem according to claim 1, wherein the at least one resource to beconfigured is an information resource.
 11. A method for resourceconfiguration in a multi-modal distributed computing system having atleast one resource capable of being configured, the method comprisingthe steps of: obtaining information associated with a mobile devicewithin the system; obtaining information associated with the system;configuring said at least one resource based upon the informationassociated with the mobile device and the system.
 12. The methodaccording to claim 11, wherein said information associated with themobile device includes contextual information associated with the needsof the user of the mobile device, the location of the mobile device, andthe environment in which the mobile device is located.
 13. The methodaccording to claim 11, wherein said information associated with thesystem includes information associated with the capabilities of deviceswithin the system.
 14. The method according to claim 11, wherein saidinformation associated with the system includes information associatedwith characteristics of the environment in which the system is located.15. The method according to claim 14, wherein said characteristics ofthe environment in which the system is located are currentcharacteristics.
 16. The method according to claim 11, wherein the atleast one resource to be configured is an interface resource.
 17. Themethod according to claim 11, wherein the at least one resource to beconfigured is a computing resource.
 18. The method according to claim17, wherein the computing resource is an application.
 19. The methodaccording to claim 18, wherein the application has multipleconfigurations and the application is configured to be most appropriatefor the environment in which the mobile device is located.
 20. Themethod according to claim 11, wherein the at least one resource to beconfigured is an information resource.
 21. A program storage devicereadable by machine, tangibly embodying a program of instructionsexecutable by the machine to perform a method for resource configurationin a multi-modal distributed computing system having at least oneresource capable of being configured, the method comprising the stepsof: obtaining information associated with a mobile device within thesystem; obtaining information associated with the system; configuringsaid at least one resource based upon the information associated withthe mobile device and the system.